多标签图像分类旨在预测图像中的所有可能标签。考虑到在每个培训图像中注释所有标签可能是昂贵的，通常将其作为部分标签的学习问题。关于部分标签学习的现有作品集中在每个训练图像只有其标签的子集注释的情况下。一种特殊情况是在每个训练图像中仅注释一个正标签。为了进一步减轻注释负担并增强了分类器的性能，本文提出了一个新的部分标签设置，其中仅标记了训练图像的一个子集，每个图像只有一个正面标签，而其余的培训图像仍保留未标记。为了处理这个新设置，我们建议一个端到端的深层网络PLMCL（部分标签动量课程学习），可以学会为部分标记和未标记的培训图像生成自信的伪标签。基于动量的新法律通过考虑更新伪标签的速度，更新每个训练图像上的软伪标签，这些标签的更新有助于避免捕获到低信心的本地最低限度，尤其是在培训的早期阶段，由于缺乏观察到的标签和培训的早期阶段对伪标签的信心。此外，我们还提出了一个信心的调度程序，以适应性地对不同标签进行易于锻炼的学习。广泛的实验表明，我们提出的PLMCL在三个不同数据集上的各个部分标签设置下优于许多最先进的多标签分类方法。

旨在恢复图像中影子区域的原始强度，并使它们与剩余的非阴影区域兼容，而没有跟踪，删除阴影是一个非常具有挑战性的问题，使许多下游图像/视频相关的任务受益。最近，变形金刚通过捕获全局像素相互作用来显示它们在各种应用中的强大能力，并且这种能力在删除阴影时非常可取。然而，由于以下两个原因，应用变压器促进阴影去除是非平凡的：1）修补程序操作不适用于由于不规则的阴影形状而导致阴影去除； 2）阴影去除只需要从非阴影区域到阴影区域的单向交互，而不是图像中所有像素之间的共同双向相互作用。在本文中，我们提出了一种新型的跨区域变压器，即CRFormer，用于去除阴影，它与现有变压器的不同之处仅通过考虑从非阴影区域到阴影区域的像素相互作用而不将图像分为斑块。这是通过精心设计的区域感知的跨注意操作来实现的，该操作可以汇总以非阴影区域特征为条件的恢复的阴影区域特征。与其他最先进的方法相比，关于ISTD，AISTD，SRD和视频阴影删除数据集的广泛实验证明了我们方法的优势。

发现新材料是一项艰巨的挑战，对人类社会的进步至关重要。基于反复试验实验和计算模拟的常规方法是劳动密集型或昂贵的，取决于专家的启发式知识，成功的方式很大。最近，通过从已知材料数据集中学习隐式知识来生成材料的生成设计模型。但是，这些模型要么适用于特定的材料系统，要么由于其未能将物理规则纳入其模型训练过程而较低。在这里，我们提出了一种基于深度学习的物理学指导的晶体生成模型（PGCGM），以实现具有高结构多样性（多达20种不同空间组）的有效生成材料设计。我们模型的高性能表明了其捕获和利用晶体的对称约束和邻居原子之间的成对原子距离约束的能力。使用数据增强和空间原子聚类和合并，我们的PGCGM模型将整体生成有效性的性能提高了700 \％以上，与FTCP相比，FTCP是最先进的结构生成器之一，与45 \％相比，我们的整体生成有效性性能提高了。我们以前的立方体模型。新生成的晶体材料在原子空间分布和组成多样性方面也显示出更高的质量。我们通过密度功能理论（DFT）计算进一步验证了新的晶体结构。 2,000个中的1,869材料成功地优化了，其中39.6％的形成能量为阴性，5.3 \％的能量库船长小于0.25 eV/原子，表明它们的热力学稳定性和潜在的合成性。 1,869个晶体结构已沉积到卡罗来纳州材料数据库\ url {www.carolinamatdb.org}。

在本文中，我们解决了一次性分段的单次无监督域适应（OSUDA）的问题，其中分段器在训练期间只看到一个未标记的目标图像。在这种情况下，传统的无监督域适应模型通常失败，因为它们不能适应目标域，以具有过度拟合到一个（或几个）目标样本。为了解决这个问题，现有的OSUDA方法通常集成了一种样式传输模块，基于未标记的目标样本执行域随机化，可以在训练期间探讨目标样本周围的多个域。然而，这种样式传输模块依赖于一组额外的图像作为预训练的样式参考，并且还增加了对域适应的内存需求。在这里，我们提出了一种新的奥德达方法，可以有效地缓解这种计算负担。具体而言，我们将多个样式混合层集成到分段器中，该分段器播放样式传输模块的作用，以在不引入任何学习参数的情况下使源图像进行体现。此外，我们提出了一种剪辑的原型匹配（PPM）方法来加权考虑源像素在监督训练期间的重要性，以缓解负适应。实验结果表明，我们的方法在单次设置下的两个常用基准上实现了新的最先进的性能，并且比所有比较方法更有效。

Waterbodies和附近相关对象的基于视觉的语义分割提供了管理水资源和处理洪水紧急情况的重要信息。然而，缺乏用于水相关类别的大规模标记培训和测试数据集可防止研究人员在计算机视野中研究水有关的问题。为了解决这个问题，我们呈现亚特兰蒂斯，一个新的水平和相关对象的语义分割的新基准。亚特兰蒂斯由5,195张Waterbodies图像组成，以及56级物体的高质量像素级手动注释，其中包括17级人为物体，18级自然对象和21个一般课程。我们详细介绍了亚特兰蒂斯，并在我们的基准上评估了几种最先进的语义分段网络。此外，通过在两个不同的路径中加工水生和非水生植物来制定新的深度神经网络水平，用于水体语义分割。 Aquanet还包含低级功能调制和交叉路径调制，可增强特征表示。实验结果表明，拟议的Aquanet优于亚特兰蒂斯的其他最先进的语义细分网络。我们声称，亚特兰蒂斯是最大的水体图像数据集，用于语义分割，提供各种水和水有关的类，它将有利于计算机视觉和水资源工程的研究人员。

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pretrained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-Shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.